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This instrumental case study documents a community-based participatory evaluation research ( CBPER ) pro- 
ject that involved a community partner, two graduate students, a faculty member, and an external funder. It 
highlights the fact that a participatory evaluation model is a viable way to conduct community-based 
research (CBR) when a community organization needs to know if the program services they are offering are 
effective. The identification of the promising aspects and shortcomings of this project advance the theoreti- 
cal and methodological understandings of the CBR field and promotes better CBR practice. 


Community-based research (CBR) is collaborative, 
change-oriented research that engages faculty mem- 
bers, students, and community members in projects 
addressing community-identified needs (Strand, 
Marullo, Cutforth, Stoecker, & Donohue, 2003). With 
its roots in action research, participatory research (for 
a detailed description between the two, see Brown & 
Tandon, 1983), and popular education, CBR is an 
important tool in engaging institutions of higher edu- 
cation with local communities (see Strand et al., 2003 
for more on the history of CBR and its relationship to 
the three stated models). Indeed, several academic dis- 
ciplines, including education (Stocking & Cutforth, 
2006), environmental health sciences (O’Fallon & 
Dearry, 2002), international education (Lewis & 
Niesenbaum, 2005), nursing (Kelley, 1995), occupa- 
tional therapy (Taylor, Braveman, & Hammel, 2004), 
planning (Reardon, 1998), public health (Israel, Eng, 
Schulz, & Parker, 2005; Minkler & Wallenstein, 
2003), social work (Rogge & Rocha, 2004), and soci- 
ology (Buroway, 2005) have acknowledged CBR’s 
contribution to the paradigm shift toward university- 
community involvement. Furthermore, the growing 
recognition of the potential of CBR to address com- 
plex community needs is recognized by the insistence 
on forming CBR partnerships contained in grant 
guidelines from federal agencies (e.g., the U.S. 
Department of Housing and Urban Development’s 
Community Outreach Partnership and the Center for 
Disease Control’s Prevention Research Centers 
Program) and other funding sources. 

With CBR, the research issue and question originate 
with the community. Often times, community organi- 
zations want to know if their programs are “working.” 


In these cases, program evaluation can become part of 
the CBR paradigm. Evaluation research is defined as 
the systematic application of social research proce- 
dures in assessing social intervention programs (Rossi 
& Freeman, 1993). Community-based researchers can 
use evaluation models such as participatory evalua- 
tion (Patton, 1997a; Stoecker, 1999) or empowerment 
evaluation (Fetterman, 1994a; Fetterman, 1994b; 
Stoecker) to help guide the community organization 
through the evaluation process. The CBR project that 
is described in this article aligns more with the partic- 
ipatory evaluation model, which is really an extension 
of the stakeholder-based model with a focus on 
enhancing evaluation utilization through the primary 
user(s)’ increased participation in the research process 
(Cousins & Earl, 1992). The principles of participato- 
ry evaluation can be summarized as follows: (a) 
involve participants at every stage of the research 
process; (b) make sure the participants own the evalu- 
ation; (c) focus the process on the outcomes the par- 
ticipants think are important; (d) facilitate participants 
to work collectively; (e) organize the evaluation to be 
understandable and meaningful to all; (f) use the eval- 
uation to support participants’ accountability to them- 
selves and their community first and outsiders second, 
if at all; (g) develop the evaluator role as a facilitator, 
collaborator, and learning researcher; (h) develop par- 
ticipants’ roles as decision makers and evaluators; (i) 
recognize and value participants’ expertise and help 
them to do the same; and (j) minimize status differ- 
ences between the evaluation facilitator and partici- 
pants (Patton; Stoecker). 

Research that is firmly based in the community is 
an important component of the community engage- 
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ment movement and with increased awareness, the 
participatory evaluation approach can be a vehicle 
through which communities can become further 
engaged in the programs which serve them. The 
practical challenges of conducting research in and 
with a community, such as unrealistic goals and time- 
frames, minimal resources, a lack of personal invest- 
ment, and methodological challenges associated with 
recruiting participants (Stocking & Cutforth, 2006; 
Strand et ah, 2003; Willis, Peresie, Waldref, & 
Stockmann, 2003), can make any CBR project diffi- 
cult to execute and complete; a CBR study whose 
purpose is to evaluate a program serving that com- 
munity is no different. Through an instrumental case 
study design 1 (Merrian, 1988; Yin, 2003), we 
describe and interpret a series of steps and sequence 
of activities that led to a particular community-based 
participatory evaluation research (CBPER) project’s 
success and failure. Specifically, we address the fol- 
lowing two questions: first, what worked and what 
did not with this CBPER project, and second, what 
lessons can be learned from a promising, but imper- 
fect, CBPER experience that might be extended to 
CBR generally? By addressing these questions, the 
case study contributes to the dialogue in the field 
regarding best practices of CBR. 

This article contains six sections. The first section 
introduces, in a narrative fashion, the CBPER project 
which is the subject of this case study; describes the 
project’s primary stakeholders; and documents how 
the project unfolded (including how stakeholders 
shaped its conception and design, collection and 
analysis of data, and the dissemination of the results). 
The second section discusses the overall impact of this 
project. The third section considers promising aspects 
of this CBPER project, while the fourth section con- 
siders shortcomings with the project. The fifth section 
identifies lessons learned for future research, and the 
sixth section discusses the implications of this case 
study to the field of CBR. In general, this case study 
shows how open dialogue can occur when academics 
and community partners listen to and value the partic- 
ular expertise each brings to the project. The result is 
the creation of a level playing field, which enables 
each partner to be full participants in each phase of the 
work. The identification of the factors that contributed 
to the success and failure of this project will inform 
professors and students as they launch their own CBR 
projects, and thus, enrich students’ education and serve 
the needs of the community. 

The Project 

Narrative 

Visitors approach an old Victorian house, nestled 
into a side street, just blocks away from one of the 


busiest avenues of the city. The painstakingly 
restored old house is the home of the Spring Institute 
and its WorkStyles program. WorkStyles is a two- 
week, intensive, pre-employment training class for 
newly arrived refugees. The occasion for this visit is 
a graduation ceremony: the 188th class of 
WorkStyles is just ending, and it is time to celebrate. 
One of the visitors writes: 

I am greeted at the door by a mix of invited 
guests, Spring Institute management and staff, 
and some of the students from the graduating 
class. The atmosphere... is one of excitement 
and anticipation. I can feel that it is a special 
day, not only for the students but also for those 
who make all they do here possible. A warm 
afternoon light fills the foyer and I am made to 
feel right at home by all who welcome me.... 

People of many races and nationalities min- 
gle together as we all begin to take our seats. I 
feel the sense of being part of something large 
and hopeful. An older gentleman from Sierra 
Leone who is one of the graduates stands in 
front of the full room and greets guests and 
graduates as the ceremony begins. He is 
dressed in beautiful African robes and cap and 
he seems to carry the dignity of his country- 
men with ease and comfort. After welcoming 
all, he opens the ceremony with a prayer and 
. . .lay [s] out the afternoon’s events. 

As the graduation ceremony begins, the observer 
continues: 

First, comes the introduction of all the gradu- 
ates. As they stand one by one, they tell a little 
about themselves and their countries of origin: 
Burma, Eritrea, Vietnam, Sierra Leone, and 
more. One by one they thank those at the 
Spring Institute who made this course possi- 
ble. Their English proficiency varies, but not 
the gratitude they share with those in atten- 
dance. Blessings are offered by many, to the 
institute, the instructors, and to all those seated 
in front of them. 

Next [comes] entertainment provided by the 
graduates... especially moving is a perfor- 
mance by a gentleman from Vietnam. He 
greets the crowd and explains in broken 
English that he spent 14 years in a refugee 
camp in the Philippines before arriving in 
America 2 months ago. He says he wants to 
sing a song he wrote about what he had been 
through to get to America. A guitar is passed 
forward, and as he begins, I can feel the perse- 
verance and pride that sustained him through 
his ordeal. I am almost moved to tears. 

Soon after, all in attendance who have yet to 
introduce themselves do so. There are people 
from other nonprofit organizations, prior grad- 
uates from the institute, institute staff and man- 
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agement, as well as individuals who have pro- 
vided funding over the years... I feel honored 
to be amongst a group of people who give so 
much to better the lives of others. Although 
attendees are of many differing races and 
nationalities, a pride and unity fill the room as 
they share in the celebration of the graduates. 
Finally, the diplomas are passed out and a few 
words of closing are offered before we all join 
together in a song that marks the end of the 
ceremony. It is time to enjoy the potluck of 
food and fortune ...provided by the graduates 
of the 188th WorkStyles class. 

This “visitor” is actually a researcher who partici- 
pated in a CBR study; the above passage is excerpt- 
ed, with minimal changes, from his field notes on the 
last day of his observation. The central purpose of the 
study was to evaluate the impact of the WorkStyles 
program upon the short-term employment of its grad- 
uates. The Colorado Refugee Services Program 
(CRSP) — the main funder of WorkStyles — requested 
and funded the evaluation study. The study was con- 
ducted under the auspices of the Colorado 
Community Based Research Network (CCBRN), 
housed at the University of Denver. The actual 
researchers — the observers who created field notes 
(such as those excerpted above), the interviewers 
who spoke to program participants, trainers, and 
community agencies, and the quantitative researcher 
who analyzed employment data of refugees who par- 
ticipated in WorkStyles as compared with those who 
did not — were five University of Denver graduate 
students. Two of the students (co-authors of this arti- 
cle) took the lead in designing and implementing the 
qualitative and quantitative portions of the study, 
under the general supervision of a professor (also a 
co-author) specializing in CBR. 

However, the composition of the study team extend- 
ed beyond the University. The Director of CRSP (fun- 
der and another co-author) played a role in the project 
by providing valuable input and feedback, especially 
at the beginning and end stages of the project. And, 
more pivotally, members of the WorkStyles staff, espe- 
cially its Employment Training Manager (another co- 
author) and trainers who are themselves refugees and 
had participated in the WorkStyles program as new 
arrivals in the U.S., were integrally involved as com- 
munity partners in every stage of the study (from plan- 
ning/design, to data collection, to reviewing and pro- 
viding substantive input on draft reports and conclu- 
sions). Thus, members of very different communi- 
ties — a state refugee service program, a university, a 
local pre-employment refugee training program, vol- 
untary agencies (VOLAGs) who case manage the 
refugees, and the refugees themselves — collaborated 
to produce a CBPER study. 


Stakeholders 

The authors of this article represent the primary 
stakeholders (see Figure 1), who are described below. 

The Community Partners 

CRSP. The Colorado Refugee Services Program 
(CRSP) is the Division of Refugee Services in the 
Office of Self Sufficiency of the Colorado 
Department of Human Services. The goal of CRSP is 
to achieve effective resettlement and rapid self-suffi- 
ciency for all refugees in Colorado. CRSP is respon- 
sible for the submission and implementation of 
Colorado’s State Plan for refugee resettlement, 
which is approved and funded by the federal Office 
of Refugee Resettlement. CRSP provides state lead- 
ership, networking, consultation, supervision, and 
monitoring on behalf of the many public and private 
agencies in Colorado that serve refugees, either as 
part of a specific caseload funded by contracts with 
CRSP, or as part of a larger mainstream caseload 
funded by other sources. The authority for the work 
of CRSP is found in the Refugee Act of 1980. 

The Spring Institute for Intercultural Learning. 
The Spring Institute is a Denver-based nonprofit 
founded in 1979 to promote intercultural sharing and 
communication as a contribution to a more peaceful 
world. Their principal goal is to demonstrate that 
national, cultural, linguistic, and ethnic differences 
are assets that foster understanding and cooperation. 
The Spring Institute provides direct services, techni- 
cal assistance, training, and consulting services 
regionally, nationally, and internationally. Programs 
include English Language Training in different busi- 
ness and community settings: Employability 
Services; Intercultural, Cross-cultural, and Diversity 
Training; Interpretation and Translation Services; 
Community Integration; and Interpreter Training. 

The Spring Institute for Intercultural Learning 
started WorkStyles in 1985. Funded through the 
Federal Refugee Social Services and Targeted 
Assistance Grant programs, WorkStyles is designed 
for non-native English speakers, specifically, to 
address the barriers to learning created by lack of 
confidence and self-esteem. To date, it has more than 
220 programs. WorkStyles is a two- week, 60-hour 
intensive course focusing on pre-employability and 
personal effectiveness skills and is designed to help 
individuals obtain, retain, and become successful in 
employment. Its training staff includes members of 
the refugee community who have first-hand experi- 
ence of the challenges of entering into the unfamiliar 
U.S. work force culture, and who themselves gradu- 
ated from the WorkStyles program. The overall pro- 
gram includes developing resumes, completing 
application forms, practicing interviews, and making 


36 


A Case Study of a Community-Based Participatory Evaluation Research (CBPER) Project 


phone calls about jobs, as well as setting goals, iden- 
tifying skills, understanding American work culture, 
and solving problems on the job. WorkStyles 
employs a variety of strategies which help to rein- 
force self-confidence by encouraging people to share 
their knowledge and experience: videotaped role 
plays, brainstorming activities, skits to demonstrate 
cross-cultural situations, small-group problem solv- 
ing, and individual exercises (Michienzi, 2003). As 
people participate in these activities, they take risks, 
and they gain confidence in their ability to use their 
verbal English skills in new and unfamiliar situa- 
tions. The program starts with the assets, skills, and 
talents that refugees bring to the job market in the 
United States and then moves out into practical 
strategies refugees can use for successfully applying 
those skills to finding and maintaining employment 
and building careers. 

The University Partner 

The university partner in this project was the 
Colorado Community-Based Research Network 
(CCBRN) housed in the Morgridge College of 


Education at the University of Denver (DU). The 
CCBRN is a clearinghouse that connects the research 
needs of community partners (i.e., schools, nonprofits, 
social agencies, and community-based organizations) 
with the skills and interests of university researchers 
(i.e,, professors and graduate students) (Stoecker et al., 
2003). Funding from local and national foundations 
and state agencies enables graduate students and pro- 
fessors to conduct up to a dozen CBR projects each 
year that address community-identified needs in the 
fields of education, public health and nutrition, and 
environmental justice. The majority of these projects 
involve qualitative, quantitative and/or mixed methods 
evaluation of new or existing programs; however, 
needs assessments, oral history projects, and 
exploratory research studies have also been undertak- 
en (for details see www.ccbm.org). 

These projects provide opportunities for graduate 
students to hone their CBR knowledge and skills 
through a pathway of CBR opportunities that 
progress from course-based, introductory, and facul- 
ty-supervised experiences to increasingly sophisti- 
cated and self-directed assignments. This pathway 


Figure 1 

The Relationship of Communities Involved in the CBPER Project 
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begins with a class in which students are introduced 
to the principals and practices of CBR and are 
required to design and implement a research project 
with a community partner. Students wishing to deep- 
en their CBR experience with community partners 
can enroll in a CBR practicum class (one quarter), 
and then participate in a paid CBR internship (up to 
a year). Paid research opportunities (such as the one 
described in this article) exist for students with the 
prerequisite experiences and skills. Finally, students 
who are considering CBR as a potential career can 
utilize the approach in master’s theses and doctoral 
dissertations (Stocking & Cutforth, 2006). 

As previously mentioned, there were members of 
multiple communities (the refugees, VOLAGS, 
CRSP, the Spring Institute, and CCBRN) that were 
involved in this project at some level; however, while 
recognizing the contributions of all stakeholders in 
this evaluation study, this article focuses on the part- 
nership between the Spring Institute and the CCBRN 
(See Figure 1) due to each partner’s central involve- 
ment in identifying the research questions, research 
design, and data collections tools, and then collecting 
and analyzing the data and disseminating the results. 

Project Conception and Deliberation 

Consistent with the principles of CBR (Strand et 
al., 2003), the initial seed of an idea for this evalua- 
tion project originated within two of the community 
partners: the CRSP and the Spring Institute/ 
WorkS tyles. The managers and coordinators of the 
WorkStyles program had begun to ask themselves 
how they could figure out whether WorkStyles was 
being successful in its mission. Specifically, they 
wondered what measures of success they could use. 
Anecdotal information led them to believe that the 
program was indeed successful. For instance, pro- 
gram graduates would come back and talk enthusias- 
tically about their lives; employers would report pos- 
itive experiences with new hires who had been 
through the program; and the VOLAGs who case- 
managed the refugees continued referring them to 
WorkStyles without reservation. WorkStyles staff 
also conducted a survey of past participants, case 
managers, and employers to understand some of the 
positive long-term benefits of the program. This sur- 
vey consisted of the WorkStyles trainers interviewing 
former students and colleagues; it yielded data that 
warranted further follow-up. However, the Spring 
Institute personnel felt that they lacked not only 
impartiality, but also the technical skills to create and 
implement specific evaluative measures so that a true 
picture of the successes (and failures) of WorkStyles 
might be presented to the outside world. 

As a first step toward an evaluation, the 
WorkStyles program manager approached the third 


author, the coordinator of the CCBRN, to find out if 
funding could be acquired to craft a meaningful and 
rigorous evaluation of WorkStyles. Somewhat coin- 
cidentally, and soon after the initial meeting between 
WorkStyles and CCBRN, the director of CRSP 
informed the WorkStyles program manager that he 
was interested in mirroring on a small scale a multi- 
ple site federal study that also was focusing on 
employment outcomes, to determine if the learning 
from and use of an evaluation would be less mean- 
ingful because of limited statistical power. Also, as 
the state agency authorized to develop and imple- 
ment the State Plan for refugee resettlement that is 
approved and funded by its federal partner (the 
Office of Refugee Resettlement), the CRSP was in a 
position to use the information gained from this small 
scale evaluation to influence public policy as it per- 
tained to the development, measurement, and posi- 
tioning of programs intended to achieve rapid self- 
sufficiency for refugees. The CRSP director wanted 
to know if WorkStyles could get such an evaluation 
done with $8,000 of available funding, and the 
WorkStyles program manager immediately referred 
him to the CCBRN. These preliminary contacts led 
to a series of meetings, attended by representatives of 
CRSP, WorkStyles, and CCBRN, to work on the pro- 
ject. CRSP’s funding would enable the DU graduate 
students associated with CCBRN to be paid for their 
time. The purpose of the meetings was to decide on 
the research questions, measures, and methods. 

Project Design 

In this series of initial meetings, as well as through 
numerous e-mail conversations, these participants 
(partners) crafted a design for the evaluation study 
with careful consideration of the needs of the funder 
(CRSP), the lead community partner (Spring 
Institute/WorkStyles), and the university (CCBRN/ 
DU). At this early stage, the funder was focused sole- 
ly on obtaining a clear picture of WorkStyles’ suc- 
cess/failure through quantitative research methods. 
Specifically, the funder was interested in answering 
the research question “Does WorkStyles impact short- 
term employment outcomes for refugees?” To meet 
the funder’s need, one of the lead doctoral students on 
the project — in the dual role of the quantitative expert 
and project manager — collaborated closely with 
WorkStyles staff to design a quasi-experimental, 
matched control group study. A matched control 
design “matches” similar subjects from two groups 
(WorkStyles and a control group of refugees who had 
not attended WorkStyles) on key attributes. The lead 
community partner’s expertise (i.e., the understanding 
of what factors or attributes were appropriate and fea- 
sible for matching groups of refugees) and the exper- 
tise of the university partner (i.e., knowledge of quan- 
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titative methodology), dovetailed nicely to produce an 
initial quantitative study design that pleased the funder 
and, in part, the community partner. 

The lead community partner, however, had some 
reservations from the very beginning about whether 
this kind of study would (or could) present a mean- 
ingful picture of WorkStyles. The WorkStyles’ man- 
ager believed that it would be difficult to find an asso- 
ciation between the WorkStyles program and early 
employment outcomes. He described his reasoning 
in the following way: 

One, it’s a two-week program that takes people 
out of the job search even though we’re teach- 
ing them job preparation skills... and two we 
don’t have any control over their job place- 
ment. ..we’re just a training organization. . .[job 
placement] is not part of our funding. 

If left to him, the WorkStyles manager would have 
chosen long-term employment outcomes for the 
study. However, this choice would have required at 
least an 18-month study, and the funder’s require- 
ment was that the study be completed within the fis- 
cal year, affording only a 9-month time frame. As 
such, the community partner concluded that they 
needed “some other data that at least gives some evi- 
dence of the [non-quantifiable] successes.” 

To that end, it was agreed that a qualitative dimen- 
sion be introduced to the project design. The funder 
was not opposed to this; in fact, he was even 
intrigued by the possibility. However, he remained 
interested primarily in the quantitative results. 
Nonetheless, the qualitative portion forged ahead. A 
second lead doctoral student with a qualitative 
research focus was added to the team, and the student 
and the WorkStyles manager collaboratively con- 
structed a qualitative design. It combined intensive 
observation of WorkStyles training sessions with the 
refugees, interviews of VOLAG representatives who 
referred refugees to the WorkStyles program, two 
focus groups of loosely matched comparison groups 
of refugees, 2 and a final focus group of the 
WorkStyles trainers. The objectives of the qualitative 
methods component of the study were two-fold: first, 
to address the question of whether WorkStyles 
helped refugees obtain early employment as well or 
better than comparable programs; and second, to pro- 
vide a more nuanced picture of the impact the 
WorkStyles program was having in aiding refugees 
to adjust to their new lives and work situations in the 
United States. The university partner was able to con- 
tribute to these objectives not only through the efforts 
of the lead qualitative doctoral student researcher, but 
also through a graduate-level CBR class, taught by 
the third author, which was serendipitously sched- 
uled to commence soon after the time at which the 


evaluation study design was being conceptualized. 
The lead qualitative researcher and the WorkStyles 
manager were invited to attend the first class meeting 
and “pitch” their project to the students. In doing so, 
they were able to recruit three additional students to 
participate in the intensive observation of WorkStyles 
training sessions, to construct field notes, and to use 
their resulting knowledge to help refine the refugee 
focus group study design. These students received 
university credit for their participation in the study. In 
this way, the capacities of the research team were sig- 
nificantly expanded. 

Thus, a mixed methods design was collaboratively 
created, in an effort to satisfy the requirements of the 
funder, the lead community partner, and university 
partner. Missing from the table at this stage of the col- 
laboration, unfortunately, were actual members of the 
refugee community; however, at the time the research 
team felt that their interests were being adequately 
represented by the WorkStyles manager. Moving for- 
ward, the collaborative study design constantly 
underwent changes due to unexpected study scenar- 
ios, such as missing data in the state’s database and, 
as we will see later, the failure of participants to show 
up to interviews and focus group sessions. In other 
words, as the university researchers and lead commu- 
nity partner immersed themselves in the collection of 
data, the research design evolved and emerged as dic- 
tated by circumstances. The emergent nature of 
research design is frequently a reality of CBR (Strand 
et ah, 2003). 

Data Collection and Analyses 

Both the quantitative and qualitative lead 
researchers, in close and constant collaboration with 
the lead community partner, moved forward sepa- 
rately, but simultaneously, to collect the necessary 
data. For the quantitative matched-control group 
study, WorkStyles staff (including refugees who were 
former WorkStyles participants) diligently compiled 
descriptive data (e.g., alien number, name, country of 
origin, date of arrival, gender) month by month for 
each of the three WorkStyles classes studied and, 
working with VOLAG staff, for a control group of 
non- WorkStyles participants (matched on factors 
such as level of English language proficiency and 
date of U.S. arrival). Detailed employment data for 
each of the study participants were obtained from a 
CRSP program analyst. (In the process of this data 
collection, suspicions about the incomplete nature of 
data regularly collected by CRSP through the 
VOLAGS were confirmed; thus, the study affirmed 
that recent improvements in the transfer of employ- 
ment and resettlement data between the VOLAGs 
and CRSP were needed to ensure the integrity of the 
state employment data.) All the necessary data were 
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sent to the lead quantitative doctoral student for 
analysis. Because these data were collected in an on- 
going fashion, frequent communication via email 
and face-to-face meetings occurred between the 
WorkStyles staff (including its refugee community 
members) and the doctoral student. 

For the qualitative part of the study, four types of 
in-depth, qualitative data were ultimately collected: 
first, 20 hours of observation of two WorkStyles pro- 
gram sessions, performed by CBR graduate students; 
second, interviews of VOLAG representatives; third, 
focus groups of WorkStyles and non- WorkStyles 
refugees; and fourth, a focus group of WorkStyles 
trainers (including three trainers who are refugee 
community members). In most CBR studies, 
unplanned events can occur (Strand et al., 2003) and 
this study was no exception. In this project, one 
unforeseen occurrence was the observation team’s 
strong recommendation to change the make up of 
focus groups, based upon what it had observed. The 
initial design (see note 2) had contemplated conduct- 
ing only two focus groups. The observation team 
objected to this design, believing that the only way 
refugees would speak candidly about their training, 
job search, and employment experiences would be if 
four separate focus groups were constituted: 
employed WorkStyles graduates, unemployed 
WorkStyles graduates, employed non- WorkStyles 
graduates, and unemployed non- WorkStyles gradu- 
ates. Responding to the observation team’s objec- 
tions, the lead doctoral student and the WorkStyles 
manager revamped the focus group design, and the 
graduate student team, using its understanding from 
the observations and the themes derived from the 
field notes, constructed protocols for conducting the 
focus groups. Again, it should be noted that the actu- 
al refugee community members of the WorkStyles 
staff or the program clients did not participate in this 
study design decision — perhaps a missed opportuni- 
ty for broadening the community collaboration. 

Next, the rationale behind having qualitative com- 
parison groups (to compare the impact of WorkStyles 
versus other comparable pre-employment programs) 
was challenged by the interview information gleaned 
from the VOLAG representatives. The VOLAGs made 
it known that there really were no comparable job 
preparation programs in the area, so they referred all 
refugees with the requisite language proficiency to 
WorkStyles. Thus, any hope that through focus groups 
the researchers would be able to make meaningful 
comparisons between the impacts of WorkStyles and 
other comparable programs was dashed. 

The focus groups themselves also did not proceed 
according to the revised plan. In an effort to foster 
trust with the prospective participants, the refugee 
WorkStyles staff members identified and contacted 


refugees to participate in the focus groups. Despite 
those staff members’ almost Herculean efforts, a 
number of the planned sessions had to be cancelled 
at the last minute due to the inability of participants 
to attend. A number of obstacles, which are not 
uncommon with the refugee population, but were not 
anticipated, came into play. These included: inflexi- 
ble employment schedules, lack of transportation, 
lack of child care, and lack of appreciation for the 
U.S. cultural expectation that appointments made are 
appointments kept. (In subsequent research projects 
with the refugee community conducted by the first 
author, these barriers have been addressed). 
Ultimately, one very rich focus group of employed 
WorkStyles graduates, and two separate interviews 
with one employed non- WorkStyles refugee 4 and one 
unemployed non- WorkStyles refugee were conduct- 
ed. Some would argue that basing conclusions on 
such a small sample size is incomplete research. This 
argument reflects the tension between conducting a 
methodologically sound study and overcoming the 
realities of conducting research with and for the com- 
munity. In this particular case, we were limited in the 
amount of time and resources we could ask our com- 
munity partner to devote to recruiting participants. 
Despite this limitation, the focus group and inter- 
views were recorded, with the consent of the refugee 
participants, and transcribed. The lead qualitative 
researcher thematically analyzed the transcripts, tri- 
angulating results with those gleaned from the obser- 
vation field notes. 

A final focus group with five WorkStyles trainers 
(dubbed “trainers’ talk’’ by the participants), three of 
whom were refugees themselves, was held to capture 
their experience with the WorkStyles process, as well 
as to member-check some preliminary interpretations 
(see below). That session was also recorded and tran- 
scribed, and the results triangulated with information 
emerging from the other qualitative data sources 
described above. 

Clearly, the unanticipated problems with the sam- 
pling design required a flexible and creative 
response. Although the team’s response may have 
detracted from the rigor of the research (a tension 
discussed in the introduction of this article), it 
ensured that data collection could be completed in a 
timely and quality manner, which is what was guar- 
anteed to the funder. 

Interpretation and Verification of Findings 

From an interpretive standpoint, the quantitative 
results revealed that completing the WorkStyles pro- 
gram did not significantly affect the short-term 
employment outcomes of the participants, as com- 
pared with the matched-control group. However, the 
qualitative findings, reflecting the central themes that 
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emerged from the qualitative data, revealed that 
WorkStyles was likely contributing to the long-term 
employment success of its graduates, as well as to their 
overall well-being in their new world. Specifically, 
three themes (culture, capabilities, and confidence, 
described as “the three C’s” in the final evaluation 
report) emerged. These themes reflect the fact that 
WorkStyles honors the cultures from which the 
refugees come, as well as teaches them about the new 
U.S. culture in which they now find themselves; that 
WorkStyles teaches practical capabilities upon which 
refugees can rely on when finding and keeping a job; 
and that WorkStyles instills confidence in refugees 
which is critical for them to succeed in their new lives. 

Verification of these results and their interpreta- 
tions occurred in two ways. First, the qualitative 
interpretation (especially the “three “Cs”) was shared 
with the WorkStyles staff — both refugee and non- 
refugee members — at the end of the “trainers’ talk” 
focus group. The lead community partner enthusias- 
tically endorsed these themes, even to the point of 
incorporating them into presentation materials about 
WorkStyles to be used at a “Training the Trainer” 
seminar the following week. Second, the lead quan- 
titative and qualitative doctoral students each created 
draft reports relating their findings, shared and edited 
each others’ work, and the quantitative doctoral stu- 
dent merged the two into a single draft report. This 
merged draft was shared with WorkStyles staff, and 
the two lead researchers conducted two face-to-face 
meetings (and wrote many emails) to obtain and dis- 
cuss the staff’s feedback. Two of the refugee staff 
members provided valuable input based upon their 
own experiences and observations. The feedback was 
carefully regarded, and led to, in some cases, signifi- 
cant changes in the report’s observations and recom- 
mendations. Thus, the experience and expertise of 
both the university and the lead community partner 
were amply reflected in the final report to the funder. 

Dissemination of the Results 

The results communicated to the funder were nei- 
ther in substance, nor in form, what had originally 
been anticipated. The university partner had under- 
stood that a largely quantitative report would be pro- 
duced for the funder, and the qualitative results 
would be more for the internal consumption and edi- 
fication of the community partner. A combination of 
the inconclusiveness of the quantitative results and 
the extremely positive experience of working with a 
committed lead community partner (as well as wit- 
nessing and appreciating the exceptional nature of 
the WorkStyles program) led the lead quantitative 
doctoral student, in her role as project manager of the 
study, to make the decision to include the entire qual- 
itative piece in the final report to the funder. 


Otherwise, she believed that the “whole picture” of 
WorkStyles’ impacts could not have been communi- 
cated; she felt that “it would have done an injustice to 
the WorkStyles program” to have restricted the final 
report to the quantitative portion of the study. 
Interestingly, the incomplete nature of the picture of 
WorkStyles presented by the quantitative results had 
been anticipated by the community partner, which 
was one of the reasons why a mixed methods design 
was pursued in the first place. Ultimately, the final 
report was forwarded to the funder, and a face-to-face 
meeting was held to discuss the report with him and 
all the evaluation stakeholders. 

Impact of the Project 

As stated above, the quantitative results revealed 
that there were no statistical differences between the 
WorkStyles group and the control group in early 
employment outcomes. This answered the original 
question posed by the funder (CRSP). However, it 
was concluded that 90 days was not enough time to 
see the employment impacts of the WorkStyles pro- 
gram. Thus, a longer study time frame was needed. 5 
The funder’s initial reaction to the quantitative study 
results was that one of the WorkStyles grants was 
possibly inappropriate as a funding source because 
its intention was to support early employment pro- 
grams. Discussion ensued regarding this point. The 
WorkStyles manager pointed out that WorkStyles’ 
purpose was to assist refugees with retention and 
long-term success, and that as a pre-employment 
training program, WorkStyles had no control over 
refugees’ job placement. That responsibility was 
already assigned to the VOLAGs. 

However, after reviewing the results and conclu- 
sions of the qualitative part of the report, the funder 
was able to see that there were longer-term integra- 
tion benefits. Integration is a topic that holds a high 
priority in nationwide refugee resettlement, and the 
report allowed CRSP to better promote and position 
WorkStyles within the range of beneficial services 
for refugees. The funder concluded that while the 
justification and outcome statements for the 
WorkStyles funding streams may need to be altered, 
there was a strong need for the program. The funder 
had recently (six months prior to the issuance of the 
report) identified an additional funding source more 
appropriate to WorkStyles’ long-term benefits that 
actually allowed the program to increase the number 
of its offerings. At the time that this manuscript was 
being written, WorkStyles had begun offering class- 
es twice per month instead of once. Therefore, as a 
result of presenting the long-term benefits of 
WorkStyles too, the overall funding for the program 
increased enough to allow for almost double the 
number of annual programs, which extends the reach 
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Table 1 

Promising Practices in This CBPER Project by Project Domain 
Project Domain What Worked 


University-Community Partnership 


Evaluation Activities 


Project Outcomes/Impact 


• Sustainable support systems 

• Collaboration and clear division of labor 

• Project team nurtured the university-community relationship 

• All partners involved in every phase of the evaluation 

• Realistic goals and project expectations set 

• Qualitative and quantitative reports combined into a more 
comprehensive final report 

• Findings verified with program staff members before dissemination 
of a more formal report 

• Final meeting convened with stakeholders to discuss the evaluation 
findings and their use 

• Findings were used by the funder and community partner 

• All partners benefited from the project 


of the program in meeting the pre-employment needs 
of many more refugees. 

Promising Practices in This CBPER Project 

Every research or evaluation project undertaken in 
collaboration with community representatives 
involves different parameters (e.g., varied goals; dif- 
fering levels of involvement, expertise, and commit- 
ment from the various stakeholders; and a wide range 
of financial resources). As such, there is no single 
recipe for success for such a project. However, the 
collaboration among the different stakeholders in this 
case study resulted in several ingredients which other 
practitioners may find helpful when planning and 
conducting their own projects in the community. 
These are listed in Table 1 . They are organized into 
three project domains: university-community part- 
nership, evaluation activities, and project out- 
comes/impacts. 

University-Community Partnership 

The university-community partnerships was one of 
the stronger aspects of this project, despite the fact 
that the current refugee program participants were 
not represented among the partners (a shortcoming 
discussed later). There were many reasons that the 
university-community partnership worked: sustain- 
able support systems between and among the part- 
ners; a sense of collaboration between the partners, 
but also a clear division of labor for participants on 
the project team; and the university-community rela- 
tionship was nurtured. 

Sustainable support systems. Due to the relation- 
ship of trust that had been built between the partners, 
every member of the research team felt he or she had 
the support of the other members. This was especial- 
ly true for the doctoral students and the WorkStyles 
manager, who received social support from each 


other, as well as academic and work-related support 
from the university professor and the director of 
refugee services. Equally important was the trusting 
relationship between the refugees and the project 
team, facilitated by active participation in the data 
gathering process by WorkStyles staff members who 
themselves were refugees. The refugees provided the 
information without which there would not have 
been a qualitative study, and such trust is hard to 
develop, and easy to shatter. 

Additionally, what was unique in this particular pro- 
ject and which undoubtedly contributed to its general 
success, was the hierarchy of responsibilities assigned 
to graduate students on the project. The quantitatively- 
oriented doctoral student was approaching the end of 
her coursework when this evaluation study took place. 
As such, she was the project manager and oversaw the 
merging of the quantitative and qualitative portions of 
the study and was ultimately responsible for producing 
the final report. The qualitatively-oriented doctoral 
student was in the middle of her coursework when the 
project was underway. Thus, while she had a lower 
level of overall responsibility, her responsibilities were 
significant. She was in charge of overseeing the entire 
qualitative portion of the study, which included, 
among other things, training the first-year graduate 
students from the CBR class. These graduate students 
had the least responsibility, yet they still contributed to 
the project by taking field notes, helping write the 
focus group protocols and, in some cases, helping con- 
duct the focus groups and interviews. This hierarchy 
was effective because it designated appropriate roles in 
the research/evaluation process to the graduate stu- 
dents based on their level of graduate school training. 
Despite this “built-in” hierarchy, the graduate stu- 
dents’ voices were equally heard during all phases of 
the project, which is consistent with CBR principles. 

Collaboration and clear division of labor. 
Collaboration, specifically the creation of equal partner- 
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ships, is one of the three guiding principles of CBR as 
well as CBPER which distinguishes them from tradi- 
tional research (Reback et al., 2002; Strand et al., 2003). 
This project was driven by a partnership structure that 
recognized the strengths and expertise of WorkStyles 
staff and doctoral students. By his own admission, the 
WorkStyles manager had limited research knowledge; 
however, he brought first-hand knowledge and practical 
expertise gained through his relationship with the pro- 
gram, with the refugees that it served, and with staff 
members who were also themselves members of the 
refugee community. This afforded him the ability to 
understand the research needed to benefit WorkStyles. 
And while the doctoral students possessed technical 
expertise and skills in quantitative, qualitative, and 
mixed research design, data analysis, and knowledge 
dissemination, the project required them to move out of 
familiar social and professional networks to work with 
a new, diverse group of people and approaches to social 
change. Because each partner recognized each other’s 
strengths and limitations and heard and valued their 
expertise, there was equal participation in the research 
process. A division of labor resulted with each person 
having one or more designated tasks with the expecta- 
tions of each party specified clearly. As one of the doc- 
toral students put it, “Everyone pulled their own weight; 
no one was a weak link. Everyone was excited. 
Everyone came to meetings with what they needed to 
come with. . . no one didn’t show up or not do what they 
were supposed to do.” Thus, the academics did not 
dominate; rather, reciprocity was demonstrated by the 
lead community partner having choice in the level of 
involvement in the project and being fully engaged in 
the creation and critique of the knowledge created. 

Nurturing the university-community relationship. In 
this particular case, not much nurturing at the begin- 
ning of the relationship was necessary because as one 
of the lead graduate students put it, “We all really 
clicked; everyone’s personality just jived from the very 
beginning.” However, even relationships that are easy 
at the beginning may eventually take some effort. In 
this particular case, the lead community partner and 
the university partner were in constant contact via 
email. Additionally, each partner was physically pre- 
sent at the site of the other. For instance, all five of the 
doctoral students who worked on this project attended 
one of the closing ceremonies (the ceremony, 
described at the beginning of this article, that cele- 
brates the completion of the WorkStyles program, as 
well as the culture from which each refugee participant 
comes). The WorkStyles manager recounts, “At one 
closing ceremony Kate [one of the graduate students 
from the CBR class] made a quilt and presented it to 
the students. She also brought in representations of her 
culture because it was Valentines Day so she brought 
in Valentines for all the students.” Additionally, the 


WorkStyles manager attended an introduction to 
research class that the university professor taught, to 
talk about this particular project. These types of 
actions extended above and beyond the responsibili- 
ties/roles set forth in the project, and helped to nurture 
the relationship between the lead community partner 
and the university partner. The WorkStyles manager 
commented, “At the level that we were all involved, 
the relationships developed more solidly, more real; 
the interactions were so much more genuine.” 

Evaluation Activities 

What worked best with regards to the evaluation 
itself were the “book end” activities — the planning 
and dissemination/utilization of the findings. 
Specifically, the funder, WorkStyles manager, and 
university researchers were all involved in the plan- 
ning process; the goal/expectation for the project was 
reasonable; the findings were verified with program 
staff members before dissemination of a more formal 
report; and a meeting with all of the involved stake- 
holders was held to discuss the evaluation findings 
and how they would be used. 

All recognized partners were involved in planning 
process. The funder, the WorkStyles manager, and 
the university representatives were all involved in the 
planning process. The identification of the research 
question — “Is WorkStyles having a short-term 
impact on the employment outcomes of its gradu- 
ates?” — came from the funder, but because the 
WorkStyles manager was also involved in the plan- 
ning, he was able to voice what he and his fellow pro- 
gram staff members hoped to learn from this CBPER 
project. As such, a multiple-outcome, mixed-meth- 
ods project was designed to meet the needs of the 
funder and lead community partner. 

Realistic goals and clear expectations. Despite the 
fact that this project had multiple outcomes, the pro- 
ject had a realistic goal and clear expectations. 
Because CBR projects strive for social change, stu- 
dents and faculty members might have lofty, ambi- 
tious goals related to ameliorating the social ills of 
our society. To compound the problem, projects 
involving university partners often have timelines 
defined by the length of time of a university quarter 
or semester, making accomplishing such elevated 
goals virtually impossible. This project had a nar- 
rowly defined, straightforward goal: to answer the 
question put forth by the funder. In this sense, our 
project was an example of a small-scale effort at 
social change (Maguire, 1993). 

Combined quantitative and qualitative findings in 
final report. On the back end of the project, a decision 
was made to combine the qualitative and quantitative 
reports to create a more comprehensive report. 
Combining the findings gave a much more complete 
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and accurate picture of the WorkStyles program. 
Additionally, the qualitative findings helped explain 
the statistically non-significant quantitative findings, 
as well as generated new research ideas about just 
how the WorkStyles program is benefiting its clients 
(by impacting longer-term integration factors). 

Verified findings with program staff members. The 
university researchers scheduled two meetings with 
the WorkStyles program staff to share the early drafts 
of the report before the dissemination of the formal 
report. This allowed them to check the findings for 
accuracy, make sure that the recommendations made 
were feasible, and incorporate any final information 
gleaned about the program, ultimately, leading to a 
more useful final report. 

Meeting to discuss evaluation findings and their 
utilization. Finally, a meeting was held with all stake- 
holders (minus the current refugee clients) to discuss 
the evaluation findings and how they might be uti- 
lized by the funder and program staff. This was defi- 
nitely one factor that allowed this project ultimately 
to have a successful impact. 

Project Outcomesllmpact 

Two project outcomes/impacts were viewed as 
successful: the findings were jointly utilized by the 
funder and community partner, and all of the partners 
benefited in some way from this CBPER project. 

Findings utilized by the funder and community 
partner. The fact that the university researchers 
worked with WorkStyles staff to refine the final 
report and that a meeting was held with all invested 
stakeholders to discuss the findings and the utiliza- 
tion of the results undoubtedly prevented the report 
from being put on a bookshelf to collect dust; rather 
the findings were utilized by both of the community 
partners (WorkStyles and CRSP). This is the goal of 
every participatory, utilization-focused evaluation 


and it moves the researcher one step closer to impact- 
ing social change — one goal of conducting research 
in collaboration with the community. 

All partners benefited from the project. All stake- 
holders — the funder, the community partner, and the 
university partner — were tmly invested in this project. 
This personal investment helped ensure this project’s 
success because each member remained motivated to 
see the project through to completion. Additionally, 
the project resulted in several benefits for all involved. 
The students experienced self-development and per- 
sonal growth as they listened to and learned from the 
community partner, applied their research skills col- 
laboratively in a community setting, and saw their con- 
tributions valued. The WorkStyles manager and 
refugee staff were satisfied with the process because 
their expertise and participation were valued during all 
or most phases of the project; consequently the final 
report addressed issues of immediate importance to 
the organization and provided understandable and use- 
ful information in a timely fashion (Kelley, 1995). 
CRSP benefited, not only by being able to better 
understand flaws in the employment data collection 
process, but also by gaining a more accurate view of 
the impact of WorkStyles, which challenged past 
assumptions and provided a new framework in which 
to position WorkStyles to better align with emerging 
integration dialogues. The refugee community itself 
benefited in a very concrete way: as a result of the pre- 
sentation to the funder of the longer-term benefits of 
WorkStyles, an increase in funding was obtained that 
now permits approximately twice as many refugees to 
benefit from the program. 

Shortcomings of This CBPER Project 

This CBPER study process, while valuable for its 
participants in many respects, did, indeed, have some 
shortcomings. Table 2 breaks down the aspects of the 


Table 2 

Shortcomings of This CBPER Project by Project Domain 

Project Domain What Did Not Work 

Project Logistics • A restricted timeframe (due to the funder's fiscal year end) 

• Limited financial resources 

University-Community Partnership • The partnership excluded current program “clients” (refugees) 

(sometimes called CBR “in the middle” (Strand et al., 2003) 

Evaluation Activities • Current program “clients” (refugees) were not included 

in the evaluation 

• Insufficient focus on program theory during the planning 

• Process-oriented or long-term outcome data were not collected 

• Team failed to anticipate barriers refugees would have to 
participate in the study 

• The quantitative and qualitative data collection and analysis efforts 
were conducted independently of each other 

• Findings from the qualitative portion were based on a small 
number of interviews conducted 
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CBPER project that did not work into the following 
project domains: project logistics, university-com- 
munity partnerships, and evaluation activities. 

These project limitations are inter-related. For 
example, the methodological limitations — e.g., the 
unduly short-term focus of the quantitative study, 
failure to collect process data, failure to focus on pro- 
gram theory in the planning stage, and the less-than- 
optimum participation level by refugees in the quali- 
tative study — can be attributed in part to external 
causes (limitations of time and funding, listed as pro- 
ject logistic shortcomings) and to internal issues 
(graduate student inexperience). From a CBR 
process vantage point, however, one central limita- 
tion looms large: the partners did not ensure that 
actual refugee program clients participated in the col- 
laborative framing of the research question and the 
formulation of the study design. In hindsight, such 
participation could easily have been procured. 
Refugee staff members at WorkStyles did engage in 
the process at the data collection/analysis and verifi- 
cation of findings phases, and they could have been 
brought in at the project’s outset as well. Perhaps if 
they had, certain subsequent difficulties encountered 
in the study — for example, the narrow scope of the 
initial research question (about which the WorkStyles 
manager expressed doubts from the start) and the 
barriers to refugee participation in the focus 
groups — might have been anticipated and articulated 
early on, and adjustments might have been made. 

Lessons Learned 

(1) A participatory evaluation is a useful CBR 
vehicle through which a community service 
provider may learn if its program is effective- 
ly meeting the needs of its clients and/or 
achieving the program’s goals. 

(2) The choice of community partner influences 
the success of the project. To increase the 
chances that a community-university partner- 
ship will yield fruitful outcomes that benefit 
both partners, a community partner must have 
the capacity to be an equal partner with stu- 
dents and university professors in the research 
process. 

(3) Graduate students selected for participation in 
CBR/CBPER projects should possess comple- 
mentary skills (e.g., quantitative and qualita- 
tive expertise) and graduate training levels 
(e.g., first-year, third-year, and fifth-year of 
graduate training). 

(4) The program’s theoretical model should be 
considered before embarking on the evalua- 
tion. This will ensure that the data collected is 


relevant and worthwhile. 

(5) Information gained from the evaluation can be 
maximized by collecting both process and out- 
comes data. 

(6) Use of a mixed-methods design (quantitative 
and qualitative methods) can also maximize 
the information gained from a project. 

(7) Early and active participation by all partners 
(including program clients) in all phases of the 
study is important and is likely to yield a more 
robust and meaningful overall process. 

(8) Every partner should be encouraged to state 
their goals/expectations for the project in the 
beginning, especially if a third-party funder is 
involved. 

(9) Care should be taken to ensure that the project 
is mutually beneficial to all parties. 

(10) A meeting should be convened at the end of 
the project to discuss the research/evaluation 
findings and how they might be utilized by the 
community partner(s). 

Conclusion 

Nationally, academics (both administrators and 
professors) are changing the engagement missions of 
their institutions in ways that embrace teaching, 
research, and outreach. Simultaneously, with dwin- 
dling public spending on social programs, agencies 
like WorkStyles are being asked to do more with 
fewer resources. CBR is continuing to emerge as a 
popular research paradigm, and the processes and 
outcomes of the project described in this article 
underline its potential to respond to the changing 
environments facing higher education and communi- 
ty organizations. As a set of research methods and 
underlying beliefs about the ways in which research 
ought to be conducted, CBR can increase the capac- 
ity of students and community partners and they can 
be more empowered to make institutional, personal, 
and social change. 

Furthermore, when case studies, such as this one, 
embrace the collective representation of participants, 
they allow for everyone’s voice to be heard and pro- 
vide a unique opportunity to gain more comprehen- 
sive insight into the progression of a generally suc- 
cessful project. The detailed documentation and dis- 
semination of the steps taken in this project advances 
the theoretical and methodological understanding in 
the field and informs better CBR practice, including 
the use of participatory evaluations, in situations that 
require the evaluation of a community program/ 
organization. Presently, there is a dearth of research 
focusing on conditions for successful and unsuccess- 
ful CBR work, and thus, the field could benefit from 
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more accounts describing what works and what does 
not work in CBR projects. Finally, it is important to 
note that this article was coauthored by all of the col- 
laborators in this project. When university and com- 
munity practitioners collaborate both on the imple- 
mentation and dissemination of CBR projects, their 
efforts may serve to enhance the credibility of CBR 
and demonstrate its effectiveness to university stu- 
dents and faculty, community members, government 
and private funders, professional associations, and 
the public at large. 

Notes 

1 Case studies are best suited for research that is 
exploratory in nature (Creswell, 1994). They seek to 
develop an in-depth understanding of complex processes 
like CBR projects, and provide inferences for successes 
and failures which can inform the field (Yin, 2003). Case 
study researchers study one or a few cases, collect mul- 
tiple sources of data, and strive for detail and depth of 
analysis (Stoecker, 2005). In this study, we drew upon 
our observations of the project as it unfolded, and relied 
on all project data, research logs, WorkStyles documents, 
field notes, email correspondence, graduate students’ 
journal entries, draft and final evaluation reports, as well 
as a formal focus group discussion involving the authors, 
which was taped and transcribed. Our analysis consisted 
of all authors reading through these sources to develop 
an overall understanding of the case, and holding a series 
of meetings to develop issues or themes about the case. 
The final step involved determining the extent to which 
these issues and themes connected with and added to the 
literature on CBR. 

2 The initial design contemplated two focus groups, to 
be divided along either the criterion of employment (one 
group comprised of employed WorkStyles graduates and 
employed alumni of other pre-employment training pro- 
grams, and the second group composed of unemployed 
WorkStyles and non- WorkStyles refugees) or the criteri- 
on of pre-employment program (one group of both 
employed and unemployed WorkStyles refugees, and a 
second group of both employed and unemployed non- 
WorkStyles refugees). The attempt to limit the scope to 
two focus groups was based on funding constraints. 

3 The WorkStyles and VOLAG staff agreed on a simple 
language assessment system based on a standardized test 
used for placement in the WorkStyles program. A 
WorkStyles participant’s score on the Basic English Skills 
Test © (BEST) was assigned a number from 1 to 3 (1 = 
beginning, 2 = intermediate, and 3 = advanced). The 
VOLAG Employment Specialists were given a list of 
WorkStyles participants with their language scores rated 1- 
3. They were then asked to identify refugees from their 
agency who did not attend WorkStyles, but who they felt 
had the same language rating. The assumption was made 
that based upon their experiences working with non-native 
English speakers, VOLAG staff would be able to rate their 
clients’ English speaking ability on this scale, but without 
the aid of the BEST assessment tool. This assumption 


proved correct and, thus, along with gender language abil- 
ity, was the third criterion for the matched group. 

4 The participant for this interview needed an inter- 
preter, and one of the WorkStyles trainers stepped in to 
provide the necessary interpreting service. 

5 However, CRSP only collects employment data on 
refugees up to 90 days. Thus, other data collection efforts 
would need to be considered to achieve a longer study 
time frame. 
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